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THE  MEASUREMENT  OP  SCHOLARLY  WORK  IN  ACADEMIC 
INSTITUTIONS:  A  CRITICAL  REVIEW  OF  THE  LITERATURE  1 

Richard  Smith  and  Fred  E.  Fiedler 
University  of  Washington ,  Seattle,  Washington 

Abstract 

This  critical  review  of  the  literature  is  concerned  with  the  measurement 
of  scholarly  work  done  by  the  faculties  of  universities  and  colleges.  Such 
measures  of  output  as  Individual  and  departmental  ratings  by  scholars,  the 
amount  of  recognition  awarded,  the  number  of  publications  written,  and  the 
number  of  citations  to  published  work,  are  discussed  and  compared.  Reference 
is  made  to  studies  that  present  empirical  findings  relating  these  measures 
to  one  another.  The  authors  conclude  that,  among  the  alternatives  discussed, 
the  citation  count  ia  the  least  biased  measure  of  scholarly  work  in  academic 
Institutions . 


*Thia  study  was  conducted  under  Contract  N0014-67-A-010 3-0012  with  the  Office 
of  Naval  Research,  Department  of  the  Navy  (Fred  E.  Fiedler,  Principal 
Investigator) . 


THE  MEASUREMENT  OF  SCHOLARLY  WORK  IN  ACADEMIC 
INSTITUTIONS:  A  CRITICAL  REVIEW  OF  THE  LITERATURE 

Richard  Smith  and  Fred  E.  Fiedler 
University  of  Washington,  Seattle,  Washington 

The  past  three  decades  have  seen  a  lomenal  expansion  of  higher 
education  in  the  United  StateB.  While  only  six  to  twelve  percent  of  the 
college-aged  youths  in  Europe  are  in  inatitutiona  of  higher  learning,  in 
the  United  States  approximately  fifty  percent  of  the  college-aged  population 
are  attending  colleges  and  universities.  It  is  very  likely  that  we  will  see 
a  further  increase  in  the  proportion  of  young  men  and  women  in  college 
within  the  next  few  years.  This  sudden  expansion  of  the  college  and  univer¬ 
sity  populations,  in  conjunction  with  the  current  tightening  of  state  and 
federal  funds  for  higher  education,  will  undoubtedly  result  in  closer 
scrutiny  of  the  effectiveness  with  which  educational  institutions  meet 
society's  goals  as  well  as  their  own  objectives.  It  is  highly  appropriate, 
therefore,,  that  social  scientists  and  educators  concern  themselves  wi  Ji  the 
adequacy  oi  our  educational  systems.  The  cornerstone  of  systematic  research 
in  this  field  that  can  lead  to  meaningful  educational  reforms  must  be  an 
adequate  basis  for  evaluating  performance.  The  present  paper  reviews  some 
measures  of  organisational  performance  in  higher  educational  institutions. 

We  shall  be  particularly  concerned  with  criteria  of  scholarly  performance 
in  graduate  teaching  and  research:  i.e.,  the  generation  and  dissemination  of 
knowledge.  These  are  clearly  two  major  purposes  of  colleges  and  univ  sides. 
While  these  institutions  have  such  other  important  functions  as  employing 
academic  and  nonacademic  personnel,  socializing  young  adults,  and  providing 
highly  specialised  manpower  for  government  and  induitry,  this  paper  confines 
itself  to  the  measurement  of  the  academic  excellence  of  university  depart¬ 
ments  and  individual  faculty  members. 

Reputational  Mea&mcs  of  Departmental  eminence.  The  earliest  major 
attempts  to  measure  the  performance  of  academic  departments  were  those 
undertaken  by  R.  M.  Hughes  (Robertson,  1928;  Hughes,  1934).  College  and 


university  professors  *»ere  asked  to  rate  the  quality  of  graduate  departments; 
20  fields  were  rated  in  1925  and  35  fields  in  1934.  Keniston  (1959)  Ob'- 
rained  similar  rankings  by  asking  department  chairmen  in  28  fields  to  rate 
sister  departments  in  other  universities.  This  was  followed  by  the  American 
College  of  Education  (ACE)  study  in  1964  (Cartter,  1966),  which  asked  a  wide 
range  of  scholars  to  rate  the  quality  of  106  university  faculties  in  29 
different  fields.  The  raters  also  judged  the  attractiveness  of  the  raduate 
programs  provided  by  these  different  departments.  The  resulting  rankings  of 
departments,  all  providing  training  at  the  doctoral  level,  comprise  at  this 
time  the  best  known  index  of  departmental  performance.  (A  revised  ranking 
of  departments  la  expected  to  appear  within  the  near  future.) 

Although  measures  of  this  type  have  some  obvious  advantages  and  also, 
as  we  shall  show  later,  moderate: validity,  they  also  have  some  obvious 
shortcomings.  The  major  limitation  is  the  high  degree  of  halo  effect  from 
which  the  department  benefits  (or  suffers)  as  a  result  of  being  part  of  a 
well-  or  poorly-known  university.  In  general,  good  departments  tend  to  be 
located  in  good  universities*  however,  some  excellent  departments  can  be 
found  in  less  highly  regarded  universities,  t-..d  some  departments  at  out¬ 
standing  universities  may  be  quite  poor.  And  even  when  the  halo  effect  is 
not  present,  it  is  possible  for  a  rater's  judgment  to  be  influenza  by 
misinformation,  hearsay,  and  his  own  personal  biases. 

A  second  major  limitation  of  reputational  measures  is  the  considerable 
time-lag  between  actual  changes  in  a  departin'-  t'a  personnel  and  teachii  ^ 
program,  and  the  reflection  of  these  changes  in  ratings  by  scholars  at  other 
schools.  Eminent  scholars  ara  notoriously  mobile,  ana  it  is  by  no  means  rare 
that  a  department  is  suddenly  stripped  of  the  four  or  five  outstanding 
scholars  on  which  its  reputation  has  been  built. 

Finally,  reputational  measures  appear  to  be  unduly  influenced  by  the 
else  of  the  department:  e  large  department  la  likely  to  be  more  visible  than 
a  small  department.  Fiedler  and  Blglan  (1969),  in  a  study  of  academic 
depart?  -s  of  the  University  of  Illinois,  found  a  correlation  of  .54  between 
AGS  rating  and  number  of  faculty  members  in  the  department.  It  may  be 
argued,  therefore,  that  reputational  measures  are  based,  at  least  in  part,  on 
departmental  visibility.  When  the  visibility  is  based  upon  the  excellence  of 
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the  research  by  members  of  the  faculty  and  the  outstanding  students  they 
have  produced,  it  quite  appropriately  contributes  to  departmental  reputa¬ 
tion.  But  if  the  visibility  is  based  upon  the  visibility  of  the  university 
or  the  sheer  size  of  the  department,  the  reputational  measures  will  produce 
spurious  results. 

Measures  Reflecting  Individual  Faculty  Member  Perforttzr.tr .  A  cumber  of 
studies  have  been  published  which  define  academic  performance  in  terms  of 
facility  productivity  at  the  individual  level.  Since  departmental  measures 
are  obtained  essentially  by  summing  or  averaging  individual  measures,  and 
since  even  the  ACE  ratings  are  in  effect  based  on  an  averaging  process  in 
the  rater'i;  head,  usxng  an  unspecified  weighting  system,  it  is  quite  im¬ 
portant  to  develop  individual  measures  of  performance.  Furthermore,  rat¬ 
ings  of  individual  faculty  members  have  the  advantage  of  making  explicit 
the  contributions  made  by  various  members  of  the  department. 

Most  ratings  of  individual  faculty  members  are  based  on  their 
publications.  It  must  be  borne  in  mind,  however,  that  the  Lasls  for  making 
these  ratings  is  less  — rect  than  would  appear  at  first  glance.  It  is  rare 
Indeed  that  the  rater  is  fully  acquainted  with  an  individual's  writing,  and 
even  more  unusual  for  the  rater  to  have  read  all  or  even  most  of  the  rater's 
publications.  Thus,  we  are  dealing  again  with  a  measure  of  reputation. 

Since  it  is  reasonable  to  assume  that  researchers  (as  practically  all 
other  people)  strive  to  be  rewarded  for  their  work,  one  way  of  measuring 
research  performance  is  to  consider  the  distribution  of  rewards  by  the 
academic  community.  The  main  reward  is  recognition  (Merton,  1957),  a  term 
which  encompasses  rewards  of  varying  importance.  The  highest  form  of  recog¬ 
nition  a  man  can  receive  is  to  have  something  named  after  him:  Euclidean 
geometry,  Newtonian  mechanics,  Levin i an  theory,  the  Wlgner  effect.  Only  a 
small  number  of  scholars  is  ever  recognised  in  this  fashion.  Considerably 
greater  numbers  are  awarded  prises  and  awards  for  their  work,  and  soma  are 
granted  membership  in  select  societies.  There  are  various  recognitions  of 
eminence  including  consultantships ,  selection  to  editorial  boards, 
scientific  panels,  or  advisory  boards,  as  wall  aa  election  to  office  in 
scientific  or  scholarly  societies. 

Crane  (1965)  related  research  productivity  and  recognition  in  the 
departments  of  Mology,  political  science,  and  psychology  of  throe  univer¬ 
sities — one  prestigious,  one  intermediate,  and  one  low  in  prestige. 
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Recognition  was  measured  by  such  honors  as  the  presidency  of,  or  membership  in, 
Certain  associations  and  societies,  honorary  degrees,  postdoctoral  fellowships, 
service  on  journal  editorial  boards,  and  other  prises.  The  measure  of  pro¬ 
ductivity  was  the  number  of  publications,  with  books  given  the  weight  of  four 
journal  articles.  The  study  shoved  that  a  man's  recognition  was  highly  re¬ 
lated  to  the  prestige  of  his  current  academic  affiliation.  Of  somewhat  less 
Importance  was  the  eminence  of  the  man's  former  academic  aponsor.  Continuity 
of  research  was  also  related  to  recognition,  provided  the  work  was  '•cndueted 
at  a  major  university.  Crane  also  found  that  56 ''percent  of  the  highly 
productive  scientists  she  studied  had  won  recognition,  whereas  only  30  percent 
of  the  less  productive  had  been  so* recognized.  She  concludes  that 
affiliation  with  a  major  university  is  more  likely  to  lead  to  recognition  for 
a  scientist  than  la  high  productivity  or  sponsor  prestige. 

Crane's  study  casts  doubt  on  the  adequacy  of  recognition  aa  a  performance 
measure.  The  prestige  of  a  man's  university  or  department  apparently 
facilitates  recognition  of  a  man's  research.  Moreover,  recognition  measures 
are  of  limited  usefulness  since  there  are  many  scholars  who  receive  little  or 
no  public  recognition  of  the  type  incorporated  in  Crane's  index — and  probably 
moat  indices  of  recognition  that  can  ba  developed.  It  is  also  likely  that  In 
some  Instances ,  personal  b!  ises  unduly  influence  the  awarding  of  recognition 
in  the  academic  community. 

Quantity  of  Research  Publications.  Llpets  (1965)  has  argued  that 
scientific  achievement  can  best  be  assessed  by  measuring  the  scientific 
content  of  research,  as  presented  in  the  scientist's  written  communications . 

In  effect,  Lipetz  calls  for  a  content  analysis  of  journo,  articles,  books, 
and  technical  reports.  A  simpler  measure  of  an  individual's  scholarly  output 
is  the  nuaber  of  articles,  books,  and  reports  hs  has  published.  A  numerical 
count  of  publication#  is  tba  most  widely  used  and  notorious  msthod  for  quickly 
asseasing  an  academician ’s productivity.  Thus,  Sonic  and  Tantnhaus  (1964) 
aaaart  that  tka  quantity  of  publication  is  tha  "standard  by  which  merit  is 
measured"  in  political  science,  uarcon  (1963)  found  correlations  of  .61  and 
.76  ba tween  publication  and  a  rating  crltarlon  of  individual  physical  scien¬ 
tists  and  biological  scientists,  respectively.  Heltzer  (1949)  found  that 
the  number  of  publications  correlated  .20  with  the  eminence  of  the  institution 
granting  the  FhD,  and  that  a  poorly  conceived  measure  of  individual  repute 
correlated  .27  with  eminence.  (The  measure  of  repute  is  poorly  conceiv-d 
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because  Che  sample  of  266  was  divided  into  only  two  categories,  high  and 
low,  and  three-fourths  of  Che  266  were  placed  in  the  low  category.)  Manis 
(1951)  found  a  correlation  of  .28  between  the  assn  measure  of  repute  and  the 
eminence  of  the  individual's  current  department  as  indicated  by  Hughes'  1934 
study,  and  a  correlation  of  .18  between  eminence  and  the  sheer  quantity  of 
an  individual's  publications.  Clark  (1957)  reported  a  correlation  of  .47 
between  Psychological  Abstract  items  (a  measure  of  quantity)  and  a  rating 
of  individual  "ewi nonce . "  On  th*  other  hand,  Tiedler  end  Biglan  (1969) 
found  a  slightly  negative  correlation  between  American  Council  of  Education 
ratings  of  a  department's  quality  and  the  average  number  of  books  published 
by  members  of  the  department  (-.18,  N»25) .  In  contrast,  the  correlation 
between  the  average  number  of  journal  articles  published  in  a  department  and 
the  department's  ACE  rating  was  slightly  positive  (.38,  N«25).  Cartter 
(1966)  found  strong  relationships  between  amount  of  publication  and  ACE 
ratings  of  political  science  and  economics  departments,  but  a  somewhat  low^r 
relationship  for  English  departments.  Re  did  not  report  relationships  for 
physical  science  c  ^artments,  ft,r  which  Pledler  and  Biglan  reported  correla¬ 
tions  around  zero.  The  latter  finding  suggests  that  the  relations  between 
reputation  and  departmental  productivity  (and  probably  individual  productivity) 
as  measured  by  number  of  publications  may  vary  widely  from  discipline  To 
discipline,  or  among  families  of  disciplines. 

A  quantity  measure  of  performance  has  its  own  limitations.  The  most 
obvious  of  these  is  that  a  poorly  conceived  paper  published  in  a  badly- 
edited  journal  will  count  as  much  ee  will  a  major  contribution  to  the  field 
which  le  published  in  e  vell-rsftretd  journal.  (Indeed,  soaie  scholars  may 
product  several  mediocre  publications  per  year,  thus  acquiring  s  very  high 
publication  count.)  Second,  it  Is  difficult  to  assign  an  s  priori  weighting 
system.  Crane  counted  e  book  as  equivalent  to  four  journal  articles. 

Maltzar,  claiming  that  an  artiela  is  equivalent  to  a  chapter,  esd  that  there 
era  on  the  average,  18  chapters  psr  book,  used  e  ratio  of  18  to  1. 

A  good  criterion  of  academic  performance  obviously  should  reflect 
quality  as  well  as  quantity.  Thl?  is  not  to  say  that  quantity  la  unimport¬ 
ant.  A  scholar  who  rarely  publishes  will  not  have  the  Impact  someone  will 
have  who  publishes  the  equivalent  ideas  in  several  different  journals 
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other  publication  outlets.  Moreover,  publication  norms  differ  widely  from 
field  to  field.  While  articles  in  many  chemical  journals  are  quite  short 
and  some  eminent  scholars  can  claim  authorship  of  several  hundred  articles, 
papers  are  more  difficult  to  write  in  such  fields  as  philosophy  or  theor¬ 
etical  physics. 

Measures  of  Quality.  Cole  and  Cola  (1967)  used  a  criterion  of  research 
output  that  seems  to  reflect  quality  more  than  the  publication  count  does. 
Unlike  Crane,  they  considered  recognition  a  reward  for  quality  rather  than 
a  direct  measure  of  It  Their  criterion  measure  is  the  number  of  citations 
an  Individual's  work  receives  in  the  literature  during  a  given  number  of 
years.  Although  they  were  not  the  first  to  describe  such  a  measure  (see 
Ruja,  1956;  Clark,  1957;  Myers  and  DeLevie,  1966;  see  also  Omstein,  et 
al.,  forthcoming*),  their  study  is  certainly  among  the  most  significant  on 
the  subject  of  research  output.  Cole  and  Cole  studied  120  physicists  in  the 
United  States,  using  the  average  number  of  weighted  citations  to  a  physicist's 
research  in  his  three  most  heavily  cited  years.  A  citation  was  given  more 
weight  if  it  was  a  reference  to  an  older  piece  of  research,  since  most  cita¬ 
tions  are  to  recent  work.  According  to  this  rationale,  a  scholar  deserves 
extra  credit  if  his  15-year-old  research  is  still  worth  quoting. 

This  citation  measure  has  several  advantages.  It  is  not  greatly 
influenced  by  quantity,  since  a  few  published  paper*  by  a  man  might  be  so 
outstanding  that  they  become  a  benchmark  for  later  research  (e.g.,  Einstein's 
small  monograph  on  his  special  theory  of  relativity;  Darwin's  Origin  of 
Species) .  Quantity  of  publication  can  be  systematically  eliminated  from  the 
measure  by  dividing  the  number  of  citations  by  the  number  of  publications 
over  a  certain  period  of  time.  An  index  of  citations  ia  relatively  easy  to 
obtain  for  certain  fields  for  which  the  Science  Citation  Index- la  avk liable, 
fcbeuth  the  routine  labor  required  for  publication  iodic**  fee  all  academic 
fields  might  require  prodigious  work.  The  index  is  based  upon  evaluations 
of  research  rather  then  on  evaluations  of  persona*  and  finally,  a  large 
nuebtT  of  a  man'*  colleagues  have  a  choice  of  citing  or  not  citing  hie 
work,  end  hence  a  voice  in  the  outcome.  Xn  e  sense,  e  citation  ia  a  rating; 
a  citation  incline  that  the  writer  considers  the  cited  work  significant 
enough  so  that  it  has  to  be  taken  into  e.  jalderetlon.  A  citation  is  there¬ 
fore  an  "unobtrusive  measure"  (Webb,  et  el.,  1966)  reflecting  the  impact  or 
significance  of  e  men's  work.  This  Is  the  case  even  when  the  reference  it 
critiqued. 

•This  work  has  Keen  cited  by  Cole  6  Cole,  1967. 
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The  measure  Hops  have  flaws,  however.  A  significant  piece  of  research 
may  not  he  recognized  for  a  considerable  period  of  tim*»  (consider,  for  example, 
Mendel's  classic  paper  on  the  genetics  of  the  aweetoea).  At  the  other  extreme, 
a  piece  of  research  may  become  so  famous  that  it  enters  the  public  domain  and 
ia  no  longer  cited  hv  name  (e.g.,  Student's  t).  Moreover,  the  differences  in 
fields  must  he  taken  into  consideration.  A  man  publishing  in  the  area  of 
analytic  chemistry  faces  different  competition  than  does  a  man  in  the  area  of 
Uittite  mythology  or  Urdu  grammar.  Finally,  a  researcher  frequently  las  a 
choit  of  sources  he  might  cite  Co  support  hia  propositions.  In  these  cases 
he  is  more  Hkelv  to  »efer  to  an  eminent  and  widely  known  authority  working  in 
a  major  university  than  to  a  relatively  unknown  researcher  at  a  small  and 
undistinguished  college,  even  though  the  latter  might  provide  somewhat  stronger 
support  for  his  case.  The  prestige  of  a  man  or  of  hia  university  is  also 
likely  to  influence  an  editor's  decision  whether  or  not  to  accept  a  paper  for 
publication.  Almost  anything  a  Nobel  laureate  might  wish  to  write  ia  likely 
to  be  published  by  a  professional  journal,  even  though  the  paner  may  not  be 
up  to  the  journal's  usual  standard?. 

The  above  criticisms  notwithstanding,  a  measure  baaed  on  citations  may 
provide  as  unbiased  a  measure  of  the  quality  of  a  man’s  work  as  we  are  likely 
to  get.  Cole  and  Cole  provide  some  supporting  data.  They  designated  the 
publication  of  30  papers  ever  a  three-year  period  a a  the  cutting  score  between 
high  and  low  quantity  of  output;  they  considered  60  citations  aa  the  cutting 
score  between  high  and  low  quality.  Tney  then  classified  physicists  into  four 
categories :  the  prolific  (high  quantity-high  quality):  the  mass  producers 
(high  quantity-low  quality):  the  perfectionists  (low  quantity-high  quality): 
and  the  silent  physicists  (low  quancity-low  quality).  Oualiry  and  quantity 
monaurei  wore  also  correlated  with  various  indirea  of  recognition.  (See 
table .  page  A. ) 

The  Coles*  data  show  that  the  quality  index  correlates  sore  highly  with 
measures  of  recognition  than  does  the  quantity  index.  The  correlation  between 
the  ACE  rank  of  a  man's  department  and  the  number  of  his  awards  wa«  .50,  while 
the.  quality  and  quantity  of  the  individual's  output  correlated  .72. 

According  to  Cole  and  Cole,  quality,  not  quantity,  la  the  main  factor 
distinguishing  award-winning  physicists  from  those  who  hare  not 
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COEFFICIENTS  OF  CORRELATION  BET'. /ESI!  QUANTITY  AND  QUALITY 
OF  RESEARCH  AM)  THREE  MEASURES  OF  RECOGNITION* 


Measures  of  Recognition 


Percent  of 

/wards  _  Community  of 


Quality  and  Quantity 
of  Research 

Prestige 
of  highest 
award 

Number 
of  awards 

Rank 

Department 

Physicists 
Familiar  with 
Individuals’ 
Research 

1.  Quantity 

.35 

.46 

.24 

.49 

2.  Number  of  papers 
per  year 

CO 

• 

.32 

.19 

.43 

3.  Quality 

.41 

.67 

.33 

.64 

•Reprinted  from  Cole  &  Cole  (1967). 
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been  so  regarded.  Quality  was  found  Co  account  for  44  per  cent  of  the  variance 
in  the  number  of  awards,  and  adding  the  factor  cf  quantity  did  not  increase  the 
amount  of  variance  accounted  for. 

Qtility  was  also  more  important  than  quantity  In  distinguishing  physicists 
in  the  top  ten  departments  from  those  elsewhere.  Xn  fact,  there  were  acre 
perfectionists  rhan  prolific  physicists  at  the  top  ten  departments  (based  on 
ACE  ranking).  These  data  do  not  sees  to  support  the  publiah-or-perish  doc¬ 
trine  when  it  is  Interpreted  primarily  in  terns  of  number  of  puolieatioas. 

As  one  might  expect,  physicists  high  on  the  quality  index  are  better 
known  by  their  colleagues  than  are  physicists  low  on  the  index.  The  nuofcer 
of  citations  to  a  physicist's  work  is  related  to  the  number  of  other  physicists 
who  have  read  at  least  some  cf  his  work  and  also  to  the  number  who  have 
heard  cf  him  but  have  not  read  his  work. 

Bayer  and  Folger  (1966)  also  utilized  a  measure  based  on  the  number  of 
citations  as  an  Index  of  quality.  They  studied  467  biochemists  who  had 
received  their  degrees  in  1957  or  1958  and  found  a  significant  correlation  of 
.21  between  quality  of  graduate  school  (based  on  ACE  ratings)  and  the  number 
of  citations.  The  correlation  between  IQ  and  number  of  citations  was  -.05, 
however.  (Obviously  this  correlation  is  highly  attenuated  by  restriction  of 
range.)  The  data  suggest  that  the  quality  of  graduate  education  may  be  im¬ 
portant  In  determining  future  research  performance,  although  the  school's 
selection  of  students  as  well  as  the  self-selection  of  applicants  for  a 
particular  school  make  such  an  interpretation  very  tenuous. 

Various  other  studies  tend  to  support  the  Cole  and  Cole  findings.  Clark 
found  a  correlation  of  .4 1  between  quantity  and  ratings  of  individual  eminence 
for  psychologists  and  a  correlation  of  .68  between  citation  count  and  eminence. 

He  found  lower  correlations  between  eminence  and  number  of  offices  held  in 
the  American  Psychological  Association  or  ausher  of  Fh.D.  students. 

Pels  and  Andrews  (1966)  conducted  a  major  study  on  the  productivity  of 
scientists.  Among  their  findings  is  am  which  is  particularly  relevant  to 
the  present  discussion:  A  correlation  of  .39  between  the  number  of  papers 
published  by  scientists  in  research  laboratories  and  their  "scientific 
contribution"  m  rated  by  their  supervisors  and  colleagues. 

A&  we  have  pointed  out  above,  Cole  and  Cole  found  that  departmental 
prestige  and  the  eminence  of  the  individual  faculty  member  are  closely  inter¬ 
related,  Which  is  cause,  and  which  is  effect  is,  of  course,  difficult  to  fathom. 
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Prestigious  departments  attract  eminent  scholars,  and  eminent  acholars  con¬ 
tribute  to  the  distinction  of  their  department.  In  addition,  the  climate  of 
an  eminent  department  undoubtedly  contributes  to  the  quality  of  research 
conducted  in  the  department.  Wilson  (1943)  studied  the  prestige  patterns  in 
the  academic  community  by  asking  a  large  sample  of  scholars  in  each  of  12 
fields  to  name  the  20  most  important  contributors  to  their  field.  He  found 
that  90.6  percent  of  the  120  leading  men  (ten  from  each  field)  were  at  ten 
highly  prestigious  universities.  This  finding  supports  Cole  and  Cole’s 
correlation  of  .50  between  the  number  of  awards  and  departmental  prestige. 

If  we  consider  prestige  a  correlate  of  the  quality  of  a  man’s  scholarly 
research,  it  is  clear  that  the  fate  of  the  individual  and  that  of  his  academic 
community  are  very  closely  interwoven. 

Discussion.  The  data  presented  in  this  paper  fairly  well  speak  for 
themselves.  We  have  attempted  to  review  and  bring  together  findings  from  key 
studies  which  examine  the  correlates  of  scholarly  research  output.  When  the 
data  are  considered  as  a  whole,  it  would  appear  that  quantity  of  publication 
is  moderately  related  to  individual  or  departmental  eminence'  that  oroductivity 
and  recognition  are  moderately  related;  that  citation  counts  correlate  well 
with  recognition  and  with  individual  eminence,  less  well  with  departmental 
prestige.  The  relationship  between  citation  counts  and  quantity  of  publica¬ 
tion  is  leas  clear:  Cole  and  Cole  report  a  correlation  of  .72,  but  Clark 
offers  .47  for  total  Psychological  Abstract  counts  correlated  with  citations 
and  only  .36  for  a  four-year  period  of  abstract  counts. 

Of  the  Indices  that  are  currently  available,  the  measures  based  on  cita¬ 
tion  seem  to  be  least  contaminated  by  such  factor*  as  the  prestige  of  the 
man’s  department  or  unive-sity,  or  the  sheer  maker  of  papers  he  has 
published .  While  measures  rsflecting  the  number  of  citations  have  their  ovm 
problems,  not  the  least  of  which  is  tha  amount  of  work  which  they  require, 
it  should  be  possible  to  reduce  considerably  some  of  the  required  effort. 

Many  sciences  are  represented  in  the  Science  Citation  Index.  For  other 
fields,  one  might  take  citations  in  standard  texts,  handbooks,  annual  reviews, 
and  journals  critically  reviewing  the  literature  as  an  acceptable  approxima¬ 
tion.  It  should  also  be  possible  to  develop  Intermediate  measures  of  output. 
Research  on  this  problem  is  currently  In  progress  by  tha  vrltare  and  their 
colleagues . 
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